Web Survey Bibliography
Title Response Rates and Response Bias in Web Panel Surveys
Year 2015
Access date 22.08.2016
Abstract
Non-probability samples, such as online panels, are increasingly accepted as “fit for purpose” for low incidence populations (e.g., pregnant women), difficult to reach populations (e.g., health care workers) and other special populations, particularly when time or cost make probability surveys infeasible. However, there is much less enthusiasm for the application of these methods in social science research for general populations. Aside from the issue of statistical generalizability, low response rates within the panel and demographic biases in the achieved samples are often cited (AAPOR 2010).
Are low response rates and demographic biases endemic to population surveys using web panels, or do they reflect the methods of particular surveys? Many web panel surveys are conducted in such a way that response rate cannot be calculated. In other cases, response rate is not reported. Further, most web surveys are not conducted to optimize response rate since sample is nearly unlimited and speed is often critically important to the client. In addition, biases in web surveys are usually identified by comparing the characteristics of the achieved sample to the population, which does not address the source of the error as the frame or the survey procedures.
This paper examines the application of two survey protocols in a general population survey conducted in the same community using a national web panel. Invitations will be sent to two Census balanced samples of 5,000 from the master panel, with the goal of achieving at least 500 completes in each sample. For the first protocol, invitations will be followed by a single reminder, an industry standard. For the second protocol, a robust reminder schedule including up to 4 reminders will be fielded over a three week period. Response rate is calculated as the proportion of invited respondents who complete the interview. Non-response bias is calculated by comparing the characteristics of responders and non-responders from their panel profile. Findings are compared across the two samples from the same community in the experiment.
Are low response rates and demographic biases endemic to population surveys using web panels, or do they reflect the methods of particular surveys? Many web panel surveys are conducted in such a way that response rate cannot be calculated. In other cases, response rate is not reported. Further, most web surveys are not conducted to optimize response rate since sample is nearly unlimited and speed is often critically important to the client. In addition, biases in web surveys are usually identified by comparing the characteristics of the achieved sample to the population, which does not address the source of the error as the frame or the survey procedures.
This paper examines the application of two survey protocols in a general population survey conducted in the same community using a national web panel. Invitations will be sent to two Census balanced samples of 5,000 from the master panel, with the goal of achieving at least 500 completes in each sample. For the first protocol, invitations will be followed by a single reminder, an industry standard. For the second protocol, a robust reminder schedule including up to 4 reminders will be fielded over a three week period. Response rate is calculated as the proportion of invited respondents who complete the interview. Non-response bias is calculated by comparing the characteristics of responders and non-responders from their panel profile. Findings are compared across the two samples from the same community in the experiment.
Access/Direct link FCSM Research Conference Homepage (Abstract) / (Full text)
Year of publication2015
Bibliographic typeConferences, workshops, tutorials, presentations
Web survey bibliography - Noncoverage & sampling (851)
- Solving the Nonresponse Problem With Sample Matching?; 2016
- HUFFPOLLSTER: Why Reaching Latinos Is A Challenge For Pollsters; 2016; Jackson, N. M.; Edwards-Levy, A.; Velencia, J.
- Predictive inference for non-probability samples: a simulation study ; 2016; Buelens, B.; Burger, J.; van den Brakel, J.
- Does the Inclusion of Non-Internet Households in a Web Panel Reduce Coverage Bias?; 2016; Eckman, S.
- Quota Controls in Survey Research.; 2016; Gittelman, S. H.; Thomas, R. K.; Lavrakas, P. J.; Lange, V.
- Scientific Surveys Based on Incomplete Sampling Frames and High Rates of Nonresponse; 2016; Fahimi, M.; Barlas, F. M.; Thomas, R. K.; Buttermore, N. R.
- Doing Surveys Online ; 2016; Toepoel, V.
- Doing Online Surveys: Zum Einsatz in der sozialwissenschaftlichen Raumforschung; 2015; Nadler, R.; Petzold, K.; Schoenduwe, R.
- Response Rates and Response Bias in Web Panel Surveys; 2015; Boyle, J.; Berman, L.; Dayton, Ja.; Fakhouri, T.; Iachan, R.; Courtright, M.; Pashupati, K.
- Characteristics of the Population of Internet Panel Members; 2015; Boyle, J; Freedner, N.; Fakhouri, T.
- Internet and Smartphone Coverage in a National Health Survey: Implications for Alternative Modes; 2015; Couper, M. P.; Kelley, J.; Axinn, W.; Guyer, H.; Wagner, J.; West, B. T.
- An Overview of Mobile CATI Issues in Europe; 2015; Slavec, A.; Toninelli, D.
- Using Mobile Phones for High-Frequency Data Collection; 2015; Azevedo, J. P.; Ballivian, A.; Durbin, W.
- Willingness of Online Access Panel Members to Participate in Smartphone Application-Based Research; 2015; Pinter, R.
- Who Has Access to Mobile Devices in an Online Opt-in Panel? An Analysis of Potential Respondents for...; 2015; Revilla, M.; Toninelli, D.; Ochoa, C.; Loewe, G.
- Who Are the Internet Users, Mobile Internet Users, and Mobile-Mostly Internet Users?: Demographic Differences...; 2015; Antoun, C.
- Optimizing the Decennial Census for Mobile – A Case Study; 2015; Nichols, E. M.; Hawala, E. O.; Horwitz, R.; Bentley, M.
- App vs. Web for Surveys of Smartphone Users: Experimenting with mobile apps for signal-contingent experience...; 2015; McGeeney, K.; Keeter, S.; Igielnik, R.; Smith, A.; Rainie, L.
- On the Go: How Mobile Participants Affect Survey Results; 2015; Barlas, F. M.; Thomas, R. K.
- Variance Estimation for Surveys from Internet Panels ; 2015; Rivers, D.
- Sensitivity Analysis of Bias of Estimates from Web Surveys with Nonrandomized Panel Selection; 2015; Beresovsky, V.
- Detecting Fraud in a Survey Sample Recruited Online; 2015; Brown, D.; Dever, J. A.; Augustson, E.; Squiers, L.
- On Climbing Stairs Many Steps at a Time: The New Normal in Survey Methodology; 2015; Dillman, D. A.
- Mobile Research Methods: Opportunities and challenges of mobile research methodologies. ; 2015; Toninelli, D. (Ed.); Pinter, R.; de Pedraza, P.
- Explorations in Non - Probability Sampling Using the Web; 2015; Brick, J. M.
- On Bias Adjustments for Web Surveys; 2015; Fan, L.; Lou, W.; Landsman, V.
- Web panel surveys – a challenge for official statistics; 2015; Svensson, J.
- Estimation with Non-probability Surveys and the Question of External Validity; 2015; Dever, J. A.; Valliant, R. L.
- Can Non-full-probability Internet Surveys Yield Useful Data? A Comparison with Full-probability Face...; 2015; Simmons, A.D.; Bobo, L. D.
- The Cathie Marsh lecture: What does the failure of the polls tell us about the future of survey research...; 2015; Sturgis, P., Matheson, J.
- Hidden Populations, Online Purposive Sampling, and External Validity: Taking off the Blindfold; 2015; Barrat, M. J.; Ferris, J. A.; Lenton, S.
- Mixed Mode Design Considerations; 2015; Hupp, A.
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2015; 2015
- Analysis of four recruitment methods for obtaining normative data through a Web-based questionnaire:...; 2015; Nolte, M. T.; Shauver, M. J.; Chung, K. C.
- Doing online research involving university students with disabilities: Methodological issues; 2015; De Cesarei, A.; Baldaro, B.
- Understanding Society Innovation Panel Wave 7: Results from Methodological Experiments; 2015; Blom, A. G.; Burton, J.; Booker, C. L.; Cernat, A.; Fairbrother, M.; Jaeckle, A.; Kaminska, O.; Keusch...
- Correcting for non-response bias in contingent valuation surveys concerning environmental non-market...; 2015; Bonnichsen, O.; Boye Olsen, S.
- An Introduction to Survey Research; 2015; Cowles, E. L.; Nelson, E.
- HUFFPOLLSTER: Pollsters Debate If Modern Surveys Can Be Trusted; 2015; Blumenthal, M.; Edwards-Levy, A.; Velencia, J.
- Using Internet to Recruit Immigrants with Language and Culture Barriers for Tobacco and Alcohol Use...; 2015; Carlini, B. H.; Safioti, L.; Rue, T. C.; Miles, L.
- Online Recruitment Methods for Web-Based and Mobile Health Studies: A Review of the Literature; 2015; Lane, T. S.; Armin, J.; Gordon, Ju. S.
- iTunes Song-Gifting is a Low-Cost, Efficient Recruitment Tool to Engage High-Risk MSM in Internet Research...; 2015; Holland, C. M.; Ritchie, N. D.; Du Bois, S. N.
- Comparing the Similarity of Responses Received from Studies in Amazon’s Mechanical Turk to Studies...; 2015; Bartneck, C.; Duenser, A.; Moltchanova, E.; Zawieska, K.
- Recruiting Online: Lessons From a Longitudinal Survey of Contraception and Pregnancy Intentions of Young...; 2015; Harris, M. L.; Loxton, D.; Wigginton, B.; Lucke, J. C.
- Recruiting for addiction research via Facebook; 2015; Thornton, L. K.; Harris, K.; Baker, A.; Johnson, M.; Kay-Lambkin, F. J.
- Can a non-probabilistic online panel achieve question quality similar to that of the European Social...; 2015; Revilla, M.; Saris, W. E.; Loewe, G.; Ochoa, C.
- Innovative Recruitment Using Online Networks: Lessons Learned From an Online Study of Alcohol and Other...; 2015; Bauermeister, J. A.; Zimmerman, M. A.; Johns, M. M.; Glowacki, P. F.; Stoddard, S. A.; Volz, E. M.
- Probabilistic Web Survey Methodology in Education Centers: An Example in Spanish Schools; 2015; Tapia, J. A., Menendez, J. A.
- Understanding Participation in a Web-Based Measurement Burst Design: Response Metrics and Predictors...; 2015; Griffin, J., Patrick, M. E.
- Facebook as a Tool for Respondent Tracing; 2015; Schneider, S. J., Burke-Garcia, A., Thomas, G.